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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set . 
forth in 37 CFR 1 .1 7(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
December 12, 2007 has been entered. 

Response to Arguments 

Applicant's arguments filed December 12, 2007 have been fully considered but 
they are not persuasive. 

Applicant argues that Ramaswamy does not disclose the elements of 
independent claims 1,12 and 23, specifically stating that, "Ramaswamy doe not teach or 
suggest the preceding limitations of claims 1,12 and 23, but instead involves a multi- 
tiered approach that, at a first stage, only outputs a target category for an utterance, but 
not a target utterance." (Remarks page 11); however the examiner respectfully 
disagrees. Ramaswamy discloses translator levels with a plurality of categories for 
each translator level, where "each category may include formal language command 
entries relevant to at least one application and may further include the step of applying 
the formal language command entries to the at least one application in accordance with 
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the input command. The at least two categories may include the step of associating a 
first portion of the entries with a given category by providing a formal command for each 
of the entries relevant for the given category." (column 2 lines 49-58). Ramaswamy 
also discloses that, "Translators 102 of a hierarchical NLU of the present invention 
includes categorized formal commands. Categories, and nested sub-categories, of the 
formal commands are defined and each formal command if assigned to one (or more) of 
the categories and stored in models 104." (column 5 lines 54-58). In Ramaswamy the 
input command is passed to translator levels, where each level including the initial level, 
interprets the utterance and assigns it to a category and corresponding formal language 
command(s) associated with that category. Thus Ramaswamy discloses, "the results 
provided at a first stage by at least one of the at least two of the plurality of handlers 
include one or more target utterance for the utterance" as stated in claims 1,12 and 23. 

Applicant also argues that, "Ramaswamy does not teach or suggest the use of 
arbitration at all, but rather simply selects the category with the highest score." 
(Remarks page 12). Applicant also asserts that, "in arbitration, a person or thing acts as 
a judge to make a choice based on discretion"(Remarks page 13), this usage of 
arbitration based on the definition provided by Webster's New World Dictionary of the 
American Language. However, the examiner continues to assert that this interpretation 
of arbitration is not supported in the specification. The specification, page 11, states 
that, "The handlers 20 decide among themselves what the intended target is by 
comparing features in the utterance to content stored in each handler to determine a 
highest score (which may include weighting and other score modification techniques, 
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which may be known in the art)". The specification also states that questions are posed 
to each handler, as stated by applicant; however the specification continues to state 
that, "each enabled handler responds in the affirmative or the negative." (page 1 3). 
Thus, winning handlers, to be used to execute commands, are determined with the use 
of binary responses to questions, and scoring amongst the handlers; these methods do 
not support the interpretation of arbitration, as argued by applicant. 

Applicant also argues that, "the cited portion of Ramaswamy discloses selecting 
a category (and not a winning handler as recited in these claims)". However the 
examiner respectfully disagrees. In Ramaswamy, the input command is passed to 
translator levels, each level then interpreting the utterance and assigning it to a category 
(winning handler) and corresponding formal language command(s) associated with that 
category, based on the probability or correct translation for each of the categories. Thus 
Ramaswamy discloses, "identifying/determining a winning handler" as stated in claims 
1,12 and 23. 

Applicant also argues that Ramaswamy does not disclose or suggest that the, 
"command is decoded in accordance with the winning handler identified by arbitration" 
since, "Ramaswamy does not output a formal language command corresponding to the 
handler (translator level) but rather then associates the selected category with a next 
level of translators and ultimately output the formal language command from a last level 
of translators." (Remarks page 14). However, the examiner respectfully disagrees. In 
Ramaswamy, the input command is passed to translator levels, each level then 
interpreting the utterance and assigning it to a category (winning handler) and 
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corresponding formal language command(s), the category determined based on the 
probability of a correct translation. The identified category (winning handler) is then 
used in the next level of continued translation. Therefore Ramaswamy discloses 
"decoding the command in accordance with the winning handler" as recited in 
independent claim 1,12 and 23. 

Applicant's arguments with respect to claims 5 and 16 have been 
considered but are moot in view of the new ground(s) of rejection. The applicant is 
encouraged to refer to the rejection of clams 5 and 16 for further explanation. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

Claims 1,4, 10-12, 15, 21-23 and 26 are rejected under 35 U.S.C. 102(b) as 
being anticipated by Ramaswamy (6,31 1 ,150). 

As per claim 1, Ramaswamy discloses a method for recognizing commands in natural 
language, comprising the steps of: 

Comparing an utterance to a plurality of handlers (column 2 lines 25-65, an input 
utterance is translated using a natural language understanding engine comprised of a 
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plurality of translator levels. Each level contains a plurality of categories with associated 
formal language commands stored as models (handlers). The categories with 
associated formal language commands are compared to the input utterance)] 

Identifying a winning handler for decoding a command from the utterance, 
wherein the winning handler is identified by arbitration between results provided by at 
least two of the plurality of handlers, and the results provided at a first stage by at least 
one of the at least two of the plurality of handlers include one or more target utterances 
for the utterance (column 2 lines 25-65, scores for the probability of a correct translation 
are determined for the categories and associated formal commands)] and 

Decoding the command in accordance with the winning handler (column 2 lines 
25-65, scores for the probability of a correct translation are determined for the 
categories, then the highest scoring category, including the associated formal 
command, are passed to the next translator level). 

As per claim 4, Ramaswamy discloses the method as recited in claim 1 , wherein the 
handlers include an enabled or a disabled state and further comprising the step of 
presenting the utterance to enabled handlers (column 8 lines 25-44), the utterance is 
compared to categories at translator levels, therefore it is inherent that they are 
enabled). 
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As per claim 1 0, Ramaswamy discloses the method as recited in claim 1, wherein the 
step of decoding further includes executing a command in accordance with the winning 
handler, responsive to the utterance (column 2 lines 25-40, scores for the probability of 
a correct translation are determined for the categories, then the category and 
corresponding formal language command having the highest score is chosen, and 
passed to the next translator level. After the last level of translation, the formal language 
command is output to control an application). 

As per claim 11, Ramaswamy discloses a computer-readable medium, tangibly 
embodying a program of instructions executable by a computer to perform method step 
for recognizing commands in natural language as recited in claim 1 (column 1 lines 40- 

46). , ... 



As per claim 12, Ramaswamy discloses a method for recognizing commands in natural 

language, comprising the steps of: 

**'.», 

Providing a plurality of handlers trained to be responsive to given utterances 
(column 2 lines 25-65 and column 3 lines 10-20, an input utterance is translated using a 
natural language understanding engine comprised of a plurality of translator levels. 
Each level contains a plurality of categories with associated formal language commands 
stored as models (handlers). The categories with the associated formal language 
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commands are compared to the input utterance, where the categories are trained from 
training data from domain inputs); 

Arbitrating against results provided by at least two of the plurality of handlers to 
determine a winning handler for an utterance, wherein the results provided at a first 
stage by at least one of the at least two of the plurality of handlers include one or more 
target utterances for the utterance (column 2 lines 25-65, scores for the probability of a 
correct translation are determined for the categories and associated formal commands); 
and 

' *!*."'" V-.. 1 * : 

Decoding the command in accordance with the winning handler (column 2 lines 
25-65, scores for the probability of a correct translation are determined for the 
categories, then the highest scoring category, including the associated formal 
command, are passed to the next translator level). 

As per claim 15, this claim has limitations similar to claim 4, and is rejected for the same 
reason. 

As per claim 21, this claim has limitations similar to claim 10, and is rejected for the 
same reason. 
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As per claim 22, this claim has limitations similar to claim 1 1 , and is rejected for the 
same reason. 

As per claim 23, Ramaswamy discloses a system for recognizing commands in natural 
language, comprising: 

A speech recognizer for decoding language and semantic information in 
utterances provided by a user (column 4 lines 61-67); and 

A dialog manager comprising a hierarchical ordering of handlers, each handler 
being trained to be responsive to decoded utterances wherein the dialog manager 
manages arbitration between results provide by the handlers to determine a winning 
handler for an utterance and decodes the command in accordance with the winning 
handler, wherein the results provided at a first stage include one or more target 
utterances for the utterance (column 2 lines 25-65 and column 3 lines 10-20, an input 
utterance is translated using a natural language understanding engine comprised of a 
plurality of translator levels. Each level contains a plurality of categories with associated 
formal language commands stored as models (handlers). The categories with the 
associated formal language commands are compared to the input utterance, where the 
categories are trained from training data from domain inputs). 
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As per claim 26, this claim has limitations similar to claim 4,.and is rejected for the same 
reason. 

v ; Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 5 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ramaswamy 

Ramaswamy discloses the method as recited in claims 4 and 15, but Ramaswamy 
does not disclose submitting the utterance to disabled container handlers to ensure 
submission of the utterance to child handlers. However, Ramaswamy does disclose 
hierarchical translators, where each level performs one portion of the translation and 
narrows the search space of formal commands for the subsequent levels (column 5 
lines 40-53). Therefore each current level determines enabled or disabled categories, 
and thus formal commands, for use in the next level. Ramaswamy is also concerned 
with developing a robust system, using techniques to improve the accuracy of the NLU 
system by checking for translation errors from previous levels (column 8 lines 25 - 
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column 9 lines 40). Submitting the utterance to disabled handlers is one method of 
checking for translation errors, since an utterance correctly translated by child handlers 
of a disabled state and incorrectly translated by child handlers of an enabled state 
indicates a possible translation error in a parent handler. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to submit the utterance to disabled container handlers to ensure 
submission to child handlers in Ramaswamy, since it would enable the system to check 
for translation errors, thus improving system performance. 

Claims 2, 3, 6, 13, 14, 17, 24 and 25 are rejected under 35-U.S.C. 103(a) as 
being unpatentable over Ramaswamy in view of Amirghodsi (4,974,191 ). 

As per claims 2, 13 and 24, Ramaswamy discloses the method as recited in claims 
1,12 and 23, however Ramaswamy does not disclose wherein the step of identifying 
includes resolving ties in the arbitration between handlers by employing a tie-breaker 
handler. Amirghodsi discloses a system that classifies objects of speech into classes 
and resolves a deadlock or tie when it occurs (column 39 lines 11-15). Amirghodsi 
discloses a natural language translation system for a human/computer interface 
(column 2 lines 1 5-1 8), and is therefore analogous art. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to resolve ties in the arbitration between handlers by employing a tie- 
breaker handler in Ramaswamy since it would enable the system to continue 
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processing after reaching a deadlock, as indicated in Amirghodsi (column 39 lines 11- 
15), thus providing quick resolution of a tie, and increased processing speed. 

As per claims 3,14 and 25, Ramaswamy in view of Amirghodsi disclose the method as 
recited in claims 2,13 and 24, however Ramaswamy does not disclose wherein the tie- 
breaker handler poses a question to a user to determine, the winning handler. 
Amirghodsi discloses a system that classifies objects of speech into classes and 
resolves a deadlock or tie when it occurs* (column 39 lines 11-15), as well as a system 
that uses questions addressed to the user to gain further information in order to process 
a user request (column 7 lines 44-55 and 59-62). Amirghodsi discloses a natural 
language translation system for a human/computer interface (column 2 lines 15-18), 
and is therefore analogous art. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to have a tie-breaker handler pose a question to a user to determine the 
winning handler in Ramaswamy, since it would enable the system to gather further 
information, as indicated in Amirghodsi (column 7 lines 59-62), which would then be 
used to determine the correct handler, thus providing quick resolution of a tie, and 
increased processing speed. 

As per claims 6 and 17, Ramaswamy discloses the method as recited in claims 1 and 
12, however Ramaswamy does not disclose further comprising the step of submitting 
unresolved utterances to winning handlers of a previous utterance for decoding. 



Application/Control Number: 1 0/701 ,784 Page 1 3 

Art Unit: 2626 

Amirghodsi discloses a system that classifies objects of speech into classes and 
resolves a deadlock or tie when it occurs by assigning the last class reference to the 
remaining objects (column 39 linesH-15). Amirghodsi discloses a natural language 
translation system for a human/computer interface (column 2 lines 15-18), and is 
therefore analogous art. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to submit unresolved utterances to winning handlers of a previous 
utterance for decoding in Ramaswamy, since it would enable the system to continue 
processing in the case of a deadlock or tie, as indicated in Amirghodsi (column 39 
lines 11-15). 

Claims 7-9, 18-20 and 27-29 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Ramaswamy in view of Ramaswamy2 ("A Pervasive Conservational 
interface for information interaction" Eurospeech 99). 

As per claims 7,18 and 27, Ramaswamy discloses the method as recited in claims 1,12 
and 23, however Ramaswamy does not disclose the step of maintaining a database of 
a history of handler selections. Ramaswamy2 discloses a system that maintains a 
database of a history of handler selections (section 2.2 Conversational System, third 
paragraph, a multimodal history captures all conversational and graphical system 
events, and keeps track of the system state). 
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Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to maintain a database of a history of handler selections in 
Ramaswamy in order to improve the natural language understanding system for the 
predictable result of enabling standard disambiguation and reference resolution, 
especially when many transactions are open, as indicated in Ramaswamy2 (section 2.2 
Conversational System, third and fifth paragraphs). 

As per claims 8,19 and 28, Ramaswamy in view of Ramaswamy2 disclose the method 
as recited in claims 7,18 and 27, however Ramaswamy does not disclose wherein the 
history includes time based ordering and ontological information. Ramaswamy2 
discloses wherein the history includes time based ordering and ontological information 
(section 2.2 Conversational System, third paragraph, a multimodal history captures all 
conversational and graphical system events, and keeps track of the system state). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use a history that includes time based ordering and ontological 
information in Ramaswamy, in order to improve the natural language understanding 
system for the predictable result of enabling standard disambiguation and reference 
resolution, especially when many transactions are open, as indicated in Ramaswamy! 
(section 2.2 Conversational System, third and fifth paragraphs). 
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As per claims 9,20 and 29, Ramaswamy in view of Ramaswamy2 disclose the method 
as recited in claims 7,18 and 27, however Ramaswamy does not disclose the step of 
resolving unresolved utterances by employing information stored in the database. 
Ramaswamy2 discloses resolving unresolved utterances by employing information 
stored in the database (section 2.2 Conversational System, sixth paragraph, the 
multimodal history is used for disambiguation and reference resolution). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use the information stored in the database to resolve unresolved 
utterances in Ramaswamy, in order to improve the natural language understanding 
system for the predictable result of enabling standard disambiguation and reference 
resolution especially when many transactions are open, as indicated in Ramaswamy2 
(section 2.2 Conversational System, third and fifth paragraphs). 



Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dorothy Sarah Siedler whose telephone number is 571- 
270-1067. The examiner can normally be reached on Mon-Thur 9:30am-5:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on 571-272-7602. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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